1. Field of the Invention
The present invention is directed to a scalable video coding system which codes video data using both frame-prediction and fine-granular scalable images. The invention has particular utility in connection with variable-bandwidth networks and computer systems that are able to accommodate different bit rates, and hence different quality images.
2. Description of the Related Art
Scalable video coding in general refers to coding techniques which are able to provide different levels, or amounts, of data per frame of video. Currently, such techniques are used by lead video coding standards, such as MPEG-2 and MPEG-4 (i.e., "Motion Picture Experts Group" coding), in order to provide flexibility when outputting coded video data.
In the scalable coding techniques currently employed by MPEG-2 and MPEG-4, an encoder codes frames of video data and divides the coded frames into a base layer ("BL") and an enhancement layer ("EL"). Typically, the base layer comprises a minimum amount of data required to decode the coded video data. The enhancement layer, on the other hand, comprises additional information which enhances (e.g., improves the quality of) the base layer when it is decoded. In operation, the encoder transmits all frames from the base layer to a receiving device, which can be a personal computer or the like. However, the encoder only transmits frames from the enhancement layer in cases where the receiving device has sufficient processing power to handle those additional frames and/or the medium over which the frames are transmitted has sufficient bandwidth.
FIGS. 1 and 2 show "scalability structures" which are currently used in MPEG-2 and MPEG-4 for the base layer and the enhancement layer. More specifically, FIG. 1 shows a scalability structure 1 which employs frame-prediction in base layer 2 to generate predicative (or "P") frames from an intra (or "I") frame or from a preceding P frame. As shown in the figure, frame-prediction is also used in the enhancement layer to generate P frames based on frames in the base layer. FIG. 2 shows another scalability structure 3 which is currently used in MPEG-2 and MPEG-4. In the scalability structure shown in FIG. 2, frame-prediction is again employed to determine P frames in the base layer. Unlike scalability structure 1, however, scalability structure 3 also uses frame-prediction in the enhancement layer to generate bi-directional (or "B") frames which, in this case, are interpolated from preceding frames in the enhancement layer and contemporaneous frames in the base layer. In general, MPEG-2 and MPEG-4 encoders use frame prediction in the manner set forth above to increase data compression and thus increase coding efficiency.
Another well-known scalable video coding technique is called fine-granular scalability coding. Fine-granular scalability coding codes the same image (e.g., a frame of video) using progressively more data each time coding takes place. For example, as shown in FIG. 3, image 4 is initially encoded using data sufficient to produce image 5. Thereafter, additional data is coded which is sufficient to produce enhanced images 6, 7 and 8 in succession.
Fine-granular scalability coding has several advantages over the frame-prediction techniques described above. Specifically, because fine-granular scalability coding can provide a wider range of enhanced images than frame-prediction techniques, fine-granular scalability coding is generally preferred in environments, such as the Internet, which have a wide range of available bandwidth. For similar reasons, fine-granular scalability coding is also generally preferred when dealing with receiving devices that have varying processing capabilities and/or bandwidth. That is, because fine-granular scalability coding produces a wide range of enhanced images, it is possible to match the appropriate image relatively closely to an amount of available bandwidth. As a result, in theory, it is possible to obtain the most amount of data for an image for a given amount of available bandwidth. On the down-side, fine-granular scalability coding does not permit the use of frame-prediction. As a result, it requires more data than the frame-prediction techniques described above and, consequently, degrades coding efficiency.
Thus, there exists a need for a scalable video coding technique which incorporates the efficiency of frame-prediction coding and the accuracy of fine-granular scalability coding.